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ABSTRACT 



A pretest-posttest control group design with random assignment, together with 
qualitative data collection and analysis, was used to investigate whether metacognitive, 
cognitive and affective-awareness cues embedded in a hypermedia programxould facilitate 
college students' near and far transfer of problem solving in biology learning. It wos 
assumed that when subjects are asked to explain reasons for their own actions during the 
problem solving, they must engage in self-reflective intermediate processes comparable to 
metacognitive processes of monitoring, evaluating, and regulating ongoing problem solving. 
Four treatment groups were used: one received metacognitive cues; one received cognitive 
cues, one received affective-awareness cues, and the control group received no cues. Results 
showed that subjects in the metacognitive group performed better on far transfer tasks 
than all other groups. The qualitative analysis indicated that metacognitive-oriented 
questions encouraged students to "stop, think and reflect" on their problem solving processes 
which in turn helped students understand the process of how the problems were solved. 
The understanding of processes students went through to solve problems helped them 
transfer what they learned to novel situations. 
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Introduction 

One of the primary goals of education is to enable individuals to develop skills which 
allow them to continue self-education throughout their lives. In the past 15 years, helping 
students develop self-regulated learning skills has been commonly known as helping 
students develop skills of learning how to learn. Self-regulated learning is defined as the 
extent to which the students are metacognitively, motivationally, and behaviorally active 
participants in their own learning processes (Zimmerman, 1986). 

The heart of self-regulation is developing the awareness and capacity for effective 
planning, monitoring, reflecting, and modifying one's own learning processes (Corno & 
Mandinach, 1983; Brown, Bransford, Ferrara, & Cam'pione, 1983). Corno and Mandinach 
(1983) further suggest that long-term exercising of self-regulated learning activities will help 
students at all levels learn how to learn and develop school aptitude and motivation. It is 
important to note that the definition of self-regulated learning encompasses metacognitive, 
cognitive, and affective processes. Flavell (1976) defines metacognition as the awareness 
and active monitoring of one's cognitive processes. In problem solving, the metacognitive 
processes monitor the selection and application of solution processes as well as ongoing 
feelings about self and task. A metacognitively aware learner is able to "apply, adapt 
and/or modify what they have learned to new tasks and across different situations" (Rowe, 
1988, p. 228). It is the understanding and regulating of their own cognitive processes that 
enable students to tackle new tasks, solve new problems, and transfer learning across 
domains (Pintrich, Cross, Kozma, & McKeachie, 1986; Glaser, 1984; Rowe, 1988; Berardi- 
Colette, 1990; Dominowski, 1990). 

Transfer is defined as the process in which students use previously learned knowledge 
and skills to successfully solve problems in new situations (Gagne & Berliner, 1988). It can 
be divided into two levels: near and far transfer. Near transfer refers to situations in which 
training situations and problem tasks are almost identical (Butterfield & Nelson, 1991; 
Salomon & Perkins, 1989). Far transfer occurs when knowledge and skills are successfully 
applied to a highly novel problem, and it requires creative use of knowledge and strategies 
learned (Clark, 1992). 

A growing body of literature supports the notion that optimal academic performance is 
strongly tied to the degree of self-regulation the learner is capable of exercising (Zimmerman, 
1990; Borkowski, et. al., 1990; Pintrich & DeGroot, 1990; Corno, 1986). As a result, 
considerable attention has been focused on the phenomenon of self-regulated learning. 
There has been much progress made in (1) identifying knowledge structures and processes 
that underlie self-regulated learning and (2) recognizing the importance of a motivational 
component in addition to metacognitive and cognitive components (McCombs & Marzano, 
1990). The integrative nature of the concept may lead to increased theoretical complexity of 
the construct and a greater understanding of learning processes. However, to use the 
concept of self-regulation as a realistic guide for everyday classroom situations, there is a 
continued need for theoretical and empirical research which shows how each component of 
self-regulation impact learning and problem solving. Further research is also needed to 
learn more about the role affective-awareness plays in contributing a better problem solving 
performance. These needs provided the theoretical basis for this study. 

Computer technology is thought to hold great potential for more effective teaching of self- 
regulated learning skills (Henderson, 1986). It has been suggested that technology-based 
instruction is likely to result in citizens who know how to learn and will continue to learn in 
the future if the program used emphasizes self-discovery of new ideas, helps learners gain 
feedback about their ideas, and encourages them to assess their own ideas (Linn, 1983). 
Hypermedia systems, a fairly recent advance in technology, provide more potential and 
advantages to learning over conventionally programmed CAI due to their greater degrees of 
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flexibility and capacity for individualized instruction (Bourne, 1990). The term "hypermedia" 
is defined as an advanced technology that combines film, video, computer graphics, sound, 
music, and text in a unified information-delivery system centered upon the personal 
computer (Paske, 1990). It is a mixture of technologies controlled by hypertext. Hypertext is 
the term that is used to describe non-sequential writing, text that branches and allows 
choices to be made by the learners (Paske, 1990). In a hypermedia system, the information 
can be organized in a non-linear manner and a topic can be explored in multiple ways (Spiro 
& Jehng, 1990). 

In addition to the theoretical problem posed earlier, two practical problems were also 
addressed in this study that involve the role of hypermedia in students 1 learning. They 
were: (a) how each component of self-regulated learning processes (metacognitive, cognitive 
and affective processes ) influenced positive near and far transfer of problem solving skills; 
and (b) how hypermedia should be used to promote self-regulated learning which might lead 
to positive transfer. 

Although previous research has shown that positive problem transfer is difficult to 
obtain (Thomas, 1974; Gick & Holyoak, 1980), studies requiring subjects to explain reasons 
for actions while solving problems have consistently shown positive transfer effects (Gagne 
and Smith, 1962; Berry & Broadbent, 1987; Berardi-Coletta, 1990; Stinessen, 1985). These 
studies typically required subjects to "state reasons" for their actions during the problem 
solving. Most of these studies dealt with college students in solving the types of problems 
involving puzzles. They all found successful near and far transfer effects. Most of their 
explanations for the positive effect on solution transfer have been limited to speculations 
that verbalization induced more planning and forced subjects to think (Gagne and Smith, 
1962), and verbalization provided the metacognitive experience needed for successful 
transfer. However, it was not clear what was responsible for the positive transfer, 
verbalization or metacognitive processes. 

Berardi-Coletta's (1990) study answered these questions. She concluded that it was not 
verbalization which produced the positive transfer effect. Rather, metacognitive processes 
enabled subjects to understand the process of how the problem was solved, which led to 
positive transfer (Berardi-Coletta, 1990). Giving reasons for one's own actions involved 
learners in metacognitive process (Berardi-Coletta, 1990). She also found in her study that 
negative self-evaluation distracted learners from focusing on the task, and thus produced 
negative effects on transfer performance. However, she did not explain how affective- 
awareness influences near and far transfer of problem solving. Further, it was not 
demonstrated in these studies if the use of self-explanation cues to induce metacognitive 
processes which were effective for puzzle problem transfer remain effective for content- rich 
problem transfer. 

It is also important to investigate whether embedding question cues in hypermedia 
system to encourage learners to self-explain their own problem-solving processes is effective 
for content-rich problem transfer because technology is likely to become an increasingly 
common component of learning in both education in schools and business training. With the 
advent of technology, especially hypertext and hypermedia which provide high levels of 
flexibility and autonomy for learners to explore and construct their own learning, it should be 
possible to assist learners to acquire self-regulatory processes that are essential for effective 
transfer of learning. 

In summary, the theoretical needs to explore how each individual component of self- 
regulated learning theory impacts learning and problem solving, together with the practical 
needs to investigate how hypermedia systems should be used to promote self-regulated 
learning and positive transfer of problem solving, provide the rationale for this study. 

The purpose of this study was to investigate 1) whether metacognitive, cognitive, and 
affective-awareness processes provoked by self-explanation cues embedded in a hypermedia 
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simulation system improves immediate near and far transfer problem solving in biology 
learning; 2) how subjects respond to metacognitive, cognitive, and affective cues; and 3) how 
metacognitive, cognitive and affective cues influence students' task design in the hypermedia 
system during the training. 

It was hypothesized that there would be no differences among groups in near transfer. 
However, subjects who received metacognitive cues were expected to perform better than 
subjects who received cognitive or affective cues and students who received no cu ^.s in 
immediate far transfer problem solving. Subjects who received cognitive and affective cues 
were expected to perform equally well on immediate far transfer of biology problem solving. 
However, subjects in cognitive and affective groups were expected to perform better than 
subjects in control groups (no cues). 



Methodology 
Experimental Design 

This study involved pretest-posttest control group design with random assignment, 
together with qualitative data collection and analysis. There were three treatment groups 
and one control group. The three treatment groups were: (1) a metacognitive group which 
responded to question cues from the computer program which asked subjects to plan ahead 
of time, monitor the process, evaluate the results and modify task design; (2) a cognitive 
group which responded to question cues that asked subjects to explain specific rules, goals, 
strategies which were related to the content of specific problems); and (3) an affective- 
awareness group which responded to question cues that asked subjects to explain their 
concurrent feelings, or to compare feelings at the beginning to those at the end. Subjects in 
the control group did not receive any cues. All of the groups received the same level of 
training, which differed only in the types of cues on which training focused. The locations of 
the cues were the same for all of the three treatment groups. The program was identical for 
all treatment groups except for the specific cues. Appendix 1 lists the content and locations 
of the cues for all three groups. 

The independent variable was the type of cues: metacognitive cues, cognitive cues, 
affective-awareness cues and no cues. The dependent variables were near and far transfer 
of problem solving. 



Subjects 

Eighty-eight pre-service elementary education majors enrolled in an introductory course 
in biology at Purdue University volunteered to participate in this study. The total enrollment 
of the class was 205, with 195 females and 10 males. The distribution of the class was 
approximately one-third juniors, one-third sophomores, with the rest a mixture of freshman 
and seniors. The study was an extra credit project option provided by the instructor. 
Originally one hundred forty students signed up for the project. But only eighty-eight were 
included in the data analysis due to several technical difficulties, which included program 
bugs and system compatibility problems. Among eighty-eight subjects, eighty-four of them 
were females and four of them were males. All of them passed content tests so that content 
knowledge would not be a factor that influenced the performance. Only 5% of the subjects 
were familiar with Macintosh computers. However, experience with computers was not 
considered a concern to the project. 
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Material? 

This research project was part of a laboratory experiment which had been found to be 
problematic for students enrolled in the class in the past. The content selected for the 
hypermedia-based training task was a series of experiments designed by the students to 
test the effects of light and moisture on isopod behavior. We will refer to these as the 
"student tasks", to avoid confusion between this experiment and the experiments which the 
students were designing. To solve these problems, subjects had to recognize the need to 
control variables and understand random behavior to verify their derived solutions, identify 
experiments which did not produce useful data, and repair them. It has been suggested by 
several researchers that the lack of metacognitive reasoning abilities might be the major 
factor inhibiting subjects from solving this type of problem successfully (Lawson, Blake, & 
Nordland, 1975; Lawson, 1979; Piaget, 1962; Lawson & Nordland, 1976). 

Training tasks were built into a hypermedia simulation program running on Apple 
Macintosh™ personal computers that linked Authorware™ (Macromedia Inc) and 
HyperCard™(Claris, Inc.). There were six options in the program menu, and subjects were 
given the freedom to select their own content, pace and paths. The program was developed 
by the authors of this paper. 

Another commercial program, Screen Recorder™(Farallon, Inc.), was also running at the 
same time when a randomly selected sample of subjects were using the simulation program. 
This program created a "record" of everything that happened on the screen, including the 
selected subjects' time on task, sequence of their selections, design of the tasks, learning 
paths, and frequency of selection of each opti -n. The purpose of this program was to collect 
qualitative data on how the different cues inLuenced subjects' responses and design of the 
experiments. 

The test of near transfer measures required subjects to work with a preset experimental 
design to test the effect of moisture on isopod behavior. The far transfer, however, asked 
subjects to design an experiment to determine which of three possible causes (peer influence, 
diet, or family environment) were responsible for an increase in hyperactivity in a 
hypothetical elementary school class. Both near and far transfer tests were developed by 
the instructor of the course. Refer to Appendix 2 for the copy of 3st. 



Procedures 

All of the students met for the lecture once per week for 50 minutes. There were also 
nine hands-on laboratory classes with about 25 students per classes. When registering for 
the lecture, subjects signed up for the lab classes based on their schedules for the semester. 
There was one instructor for the lecture and nine teaching assistants (TAs) for the lab 
divisions. Each hands-on lab division met for two 90 minutes sessions/week, and there 
were always two teaching assistants present. 

The project was divided into three phases, pre-treatment (four weeks), treatment ( two 
weeks) and post-treatment (one week). The pre-training was conducted because it has been 
suggested by Berry and Broadbent (1984) that concurrent verbalization or self-explanation 
may only have a significant effect when accompanied by pre-task training on how to respond 
to the questions. 

During the first week of the pre-treatment phase, TAs were introduced to the project by 
the researchers. The purposes of the project, time required of TAs for help, and overall plan 
and scope of the project were all explained. They were also asked to go through the 
computer program as well. The researchers went to the lecture to explain the project to all 
of the students. The policy for extra credit was explained as well as the learning benefits 
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students might obtain by participating in the project. They were told that this experience 
could help them to learn to solve problems better, which could be applied to other course 
projects they would be assigned. They were also told that training would be provided by the 
researchers and TAs so that they would know how to use the program when they came to 
the computer lab. This was done to alleviate possibilities of student unfamiliarity and/or 
anxiety about the use of computers. 

During the second and third week of the pre-training phase, subjects completed a hands- 
on isopod lab experiment during their regularly scheduled lab divisions. At the same time, 
the sign-up sheets for the computer lab schedules were posted so that subjects would have 3 
weeks to sign up for the computer labs. Each subject was required to sign up for two 
computer sessions with 2 hours each session. TAs were instructed by the researchers on 
how to train subjects from different treatment groups to respond to different cues. The 
instructions for the training were given to all of the TAs. Students were randomly assigned 
within divisions to each treatment group by the researchers, and the TAs were also informed 
of the names within each treatment group. 

During the fourth week of the pre-training phase, student training on how to work with 
the programs and how to answer cue questions were conducted by the researchers and TAs 
in their regular lab sessions. The random assignment within divisions gave approximately 
7 subjects per treatment group within each division. 

During the treatment phase, eighteen Mac Ilex computers in a computer lab were used. 
Each computer was equipped with the four versions of the program corresponding to the 
three treatment groups and one control group. Each version required subjects to enter their 
own passwords to access a specific treatment. The password was given to the subjects 
during the pre-training session. By assigning the passwords to each student, the 
treatments were effectively isolated, preventing any opportunities for mixing the treatments. 
The researchers supervised the lab across all sessions to make sure that everything ran 
smoothly. 

During the post treatment phase, subjects were informed about the real nature of the 
project and were told that treatment was manipulated. They were encouraged to practice 
with different versions of the program so that everyone was exposed to the same versions of 
the program to reduce the possible effects caused by the manipulation in their performance 
on the course final exams. 



Data Collections a nd Analysis 
The data collection was divided into three stages, pretest data collection, on-line data 
collection and posttest data collection. Three weeks before the treatment, all of the subjects 
who signed up for the project were given pretests on near and far transfer problem solving. 
The on-line data was collected by Screen Recorder™(Farallon, Inc.) running at the same time 
when selected subjects were working in the program, as well as information which the 
program itself recorded to disk 

(student responses to the cues and time on task). Three subjects from each treatment group 
and the control group were selected randomly. No subjects were interviewed during the 
treatment time. 

As soon as the subjects finished two computer lab sessions, they were given the 
posttests on near and far transfer. The content was exactly the same for pre and posttests 
except that the orders of the questions had been changed. Eight subjects from each of the 
treatment and control groups for a total of 16 were selected for interviews right after they 
finished the near and far posttests. Five questions were used to direct the interview, and 
the same questions were asked across groups. They were 1) how subjects responded to the 
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different cues; 2) how the cues affected the process of on-line task problem solving, 3) why 
some subjects did not respond to the cues; 4) whether the subjects asked themselves the 
same questions during the transfer problem solving; and 5) how the experience affected 
subjects' confidence and task interests. Refer to Appendix 4 for the sample protocols from 
each group. 

The data was analyzed both quantitatively and qualitatively. One-way ANOVA and 
post-hoc Newman-Keuls were used to measure posttest differences among groups. Paired 
sample t-tests were used to measure the pre and posttests differences within groups. A 
random sample of 6 protocols from each group were coded for the cue response content by 
two independent raters. Interrater reliability was checked to be sure that subjects in the 
different treatment groups were in fact responding to the type of information that was 
requested in the question cues. The qualitative data were analyzed within each group to 
find the patterns of subjects' responses and learning processes based on their responses to 
the interviewer, and the student task designs were analyzed through a constant comparison 
process for any subgroupings. 



Results 

Quantitative Findings 

To test the main hypothesis that all of the groups would perform equally well on 
immediate near transfer test, and that the metacognitive group would outperform all of the 
other groups in immediate far transfer test, analysis of variance (ANOVA) was used to 
compare performance across the groups on near and far transfer measures. Refer to Table 1 
for the means and standard deviations for the pre and post measures on dependent 
variables among groups. There was no significant mean score differences among groups on 
the pretest of near or far transfer. Neither were significant mean score differences found 
among groups on posttest near transfer and the average time spent on the training task. 
However, the ANOVA for post far transfer tests indicated there was a statistically 
significant main effect among groups: F (3, 84) = 5,95, p <.001. Student-Newman- Keuls 
(SNK) was chosen for post hoc paired comparisons contrasting group means. Results of a 
post hoc SNK showed the order of the group means (in descending magnitude) in post far 
transfer (Refer to Table 2). Means which were not underlined by the same line were 
significantly different from each other. The finding suggests that the metacognitive group 
outperformed all of the other groups in posttest far transfer. The cognitive, affective- 
awareness, and control groups were not significantly different from each other. The main 
hypotheses were supported. 
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Table 1: Individual group means and standard deviations for dependent 
variables of pre and posttests on near and far transfer: 



Variable 


Group 


N 


Mean 


Std Dev 


Pre-Near 


Affective 


21 


15.619 


2.706 




Cognitive 


20 


15.550 


1.8129 




Controi 


23 


16.239 


2.1313 




Metacog. 


20 


15.925 


2.7012 


Pre -Far 


Affective 


21 


2.380 


1.5961 




Cognitive 


20 


1.950 


1.5035 




Control 


23 


2.043 


1.637 




Metacog. 


20 


2.150 


1.785 


Post-Near 


Affective 


22 


16.59 


2.338 




Cognitive 


21 


16.69 


2.337 




Control 


24 


16.39 


3.65 




Metacog. 


21 


18.09 


1.505 


Post-Far 


Affective 


22 


2.318 


1.728 




Cognitive 


21 


2.523 


1.364 




Control 


24 


1.708 


1.366 




Metacog. 


21 


3.476 


1.123 


Time 


Affective 


15 


77.46 


24.12 




Cognitive 


15 


70.00 


18.40 




Control 


17 


80.29 


33.79 




Metacog. 


17 


67.58 


18.12 



Table 2: Results of the Post Hoc SNK contrasting group means on post far 
transfer test: 



Meta Cognitive Affective Control 



M 3,476 2.524 2.318 1.708 

The differences between pre-post mean scores, standard deviations, T scores, and p 
values on near and far transfer within groups are listed in Table 3. The results indicate 
that metacognitive group was the only group which had significant differences between pre 
and posttests on both near (p <.01) and far transfer (p < .05). The closest to significance in 
pre and posttests on far transfer was found within cognitive group 

(p = .07). Affective and control groups did not have significant differences between pre and 
posttests on both near and far transfer. 
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Table 3: The differences between post-pre mean scores, standard deviations, 
T scores, and p values on near and far transfer within groups 



Groups 


Means (post-pre) 


Std Dev. 


T Score 


P-values 


Affective- 










awareness 










Near 


.928 


3.15 


1.35 


.192 


Far 


-.095 


1.94 


-.01^4 


.824 


Cognitive 










Near 


1.025 


3.53 


1.29 


.210 


Far 


.65 


1.56 


1.85 


.07 


Control 










Near 


.260 


4.77 


.262 


.79 


Far 


-.260 


1.91 


-.654 


.51 


Metacog. 










Near 


2.125 


3.30 


2.87 


.0096 


Far 


1.4 


2.32 


2.69 


.0144 



Qualitative Findings 

Immediate post task interviews, on-line data collection, and student task design 
analysis were used to answer the two qualitative research questions: 1) how subjects 
responded to metacognitive, cognitive and affective cues, and 2) how metacognitive, cognitive 
and affective cues influenced the students' task designs in the hypermedia system during 
the training. 

To answer the first question, coding consisted of classifying students' responses to each 
question cue into one of the 3 levels of focus. The following is a list of the categories used in 
the coding of students' responses to the different question cues embedded in the 
hypermedia-based simulation program: 

A. Process-Oriented Responses: 

1. Planning (one step ahead, two steps ahead, etc.) 

2. Monitoring (recognize right/wrong choices and why) 

3. Subgoaling ("I decided to test that after I finished this because...") 

4. Evaluative ("This experiment seems to be working because...") 

5. Modifying ("I realize that I need to replicate what I have got in the first trial to draw 

more reliable conclusions.") 

B: Cognitive-Oriented: 

1. Stating specific rules for the problem ("the experiment needs to control variables, etc.") 

2. Stating goals of the problem ("test how light and moisture affect isopod behaviors") 

3. Recognizing the current state of the problem solving (" I got this answer right"; "my 
second experiment yielded right answer") 

C: Personal Feeling-Oriented: 

1. Positive ("I feel OK, fine, confident, etc.") 

2. Negative ("I feel frustrated"; "It was too confusing") 

To look at how subjects in the different treatment groups were responding to the 
questions that were asked in the program, a random sample of 6 protocols from each of the 
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treatment groups was coded for the content by two independent raters according to the 
categories classified above. The summary consisted of the number of responses judged to be 
a member of each category for each question cue by each subject. A grand summary for each 
treatment was then calculated by each rater. The interrater reliability indicated 90% 
agreement. The differences were resolved by discussing the responses in question and 
deciding upon agreed-on judgments. 

The total percentage of the responses made in each major category by each group was 
then calculated and compared to find out whether any cf the groups differed in terms of the 
content oi' their responses. It appears that each group focused on the appropriate level of 
information requested by the questions (see Table 4). The affective subjects gave 96.5% 
feeling-oriented statements, 3.5% process -oriented statements and almost no cognitive- 
oriented statements. Among feeling-oriented statements, 65% were rated as positive and 
35% of them were negative statements. Cognitive subjects gave responses that fell into the 
problem-oriented category (rules, goals, and problem state) 85.7% of the time. 14.3% of the 
time they made process-oriented responses and they almost made no feelings-oriented 
statements. The metacognitive subjects gave 72% of the responses that fell into the process 
category, 21% of the responses fell into problem or cognitive level and 7% fell into feeling 
levels. 



Table 4: Category percentages for each group 



Affective Cognitive Metacognitive 

Process 3.5% 14.3% 72% 

Cognitive 0 85.7% 21% 

Personal 95.5% 0 7% 

Total 10Q% 100% 100% 



In further analysis of the interview data, it was found that metacognitive subjects were 
involved more in the reflective processes. When they reached the solution, they were not just 
satisfied with it, but went further to find out why it was right or wrong. For example, when 
asked to summarize how the questions affected the whole process , one student responded 
"I think that they really made me go back and look at it and check myself to see that I was 
doing them right and really looked at my experiments to see how the results were turned out 
.Otherwise, I might just finish and not go over it" (emphasis is the authors' ). Another 
student said that these questions helped her to " question further about [her] conclusions, 
like why. They made [her] make sure that they were valid enough". When this same 
student was asked if she would do anything differently if she were not asked the questions, 
she responded " Not really. I probably would not understand the processes I went through 
this well. I would just be satisfied that I got them right" In fact, most people from the other 
3 groups were just happy to get the right answers and stopped there. One student in the 
cognitive group summed this up well . She said: " I did not really ask myself anything at the 
end. I was just glad. I knew I was correct." 

Additionally, it appeared that while solving far transfer problems, metacognitive 
subjects continued to ask themselves questions similar to those presented in the computer 
program. This was not indicated by other groups of subjects. One student in the 
metacognitive group stated "Yes. I did go through the similar processes. I asked myself at 
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the first place why I knew my results and setting up the experiments are right, why was I 
confident that I got a certain part right because I remembered my computer experiences". 
This response is characteristic of 11 of the 14 metacognitive students interviewed. 

Affective subjects reported that awareness of on-task feelings helped them realize how 
they were doing in the problem solving process. If they were aware that they were 
frustrated or nervous, they felt that there must be something wrong with their performance. 
But many of them reported that they did not know what was wrong or how to fix it. ("It 
made me think about how I was solving the problem by realizing how I was feeling at that 
moment. But the question did not help me solve the problem better. Because they did not 
help me find out where is my problem and how to solve it.") Their responses told us that 
the awareness of emotion state is not sufficient to enhance the problem solving performance. 

During the interview of control subjects, most of them indicated that they were very 
confused all the time and they needed more help. They felt that decision-making was the 
toughest part in the whole process. They were glad that feedback was given at the end to 
help them realize that they got right answers. "I feel very confused the whole way through. I 
wish I had more help. I was not sure if I was right or wrong. The decision-making part was 
very hard on me." Overall, subjects across groups reported increased interest and confidence 
in using computers. Many of them said "I like computer experiments. It saves time and 
mess. (metacognitive group)..."; "I like computers and I would feel more confident if I have to 
use it again.. (Cognitive group)."; "I think that it was very neat that I could visually see what 
was happening in my experiments. I become more confident in using computers after this 
hard (control group)"; and "I like computers and wish to have more experience with it. 
(affective group)". 

To answer the question of how metacognitive, cognitive and affective cues influence 
students' task design in the hypermedia system during the training, a constant comparison 
analysis was performed . The students were asked to diagram their designs on paper 
immediately after finishing the computer program. Figure 1 shows a visual representation 
of the process of forming the groups. In the end, the designs fell into one of four (4) large 
groups: designs exactly like those on the program; designs which did not show any 
separation of variables; designs which showed a separation of variables, but did not test the 
two variables against each other; and designs which tested the variables against each other. 
The group which showed separation of variables was subdivided by number of variables 
tested ( 1, 2 or 3) and whether or not those in the 2 variable category ran replications or not. 
This resulted in four subcategories in this group. The group which showed some testing of 
the variables against each other was also subdivided into those who only compared the two 
variables, and those who both isolated the variables and compared them. This last group 
was subdivided further as to whether the testing of the variables against each other was 
correctly designed or not, and if the correctly designed one ran replications or not. This gave 
a total of four subgroups in this category, making a total often possible groups of designs 

In each of these ten groups, the students were clumped as to which of the four 
experimental groups they participated in. Numbers of students in each group were 
converted to the percentage of the total number of students from theii treatment group, so 
co^Darisons could be made across treatment groups. See Table 5 for these results. 

xhe ideal task design would be to both isolate variables and correctly tested them 
against each other( group 10) . If we do not consider whether the student carried out 
replications, then 19% of the cognitive students are seen in this group, 6.60% of the affective 
students, 4,7% of the control students and 4.5 % of the metacognitive students. 
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FIGURE 1: 

A Visual Representation of the Resulting Groups for Analysis of Student-reported Designs 




ALL. DESIGNS 



J 



(same As Paula's Design } 



( Different horn Pauls 's design ^ 




(^No Sepesation ->f Variables ) 



Separation of Variables ^ 




( ONLY geparatedVaxiabler^) 



Cormpased the 2 Variables 



( 1 Variable ) V^iables ) ( 3 ) CCombiOfid Only ) (Sepa^&Combi 




Replication 3 C NO Replication ^ 




(^Combined cowectly^) (Combined WlOU 
C Replication j ( No replication ) 



Table 5: Comparison of Group Compositions for Design Analysis: 



group 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


control 


4.70 


4.70 


9.50 


9.50 


47.00 


0.00 


9.50 


4.70 


4.70 


0.00 


affective 


0.00 


13.30 


13.30 


6.60 


33.0 


13.30 


13.00 


0.00 


6.60 


0.00 


cognitive 


4.70 


4.70 


19.04 


19.04 


28.50 


0.00 


0.00 


0.00 


9.50 


9.50 


metacog 


9.00 


4.50 


4.50 


27.00 


32.00 


4.50 


0.00 


4.50 


4.50 


0.00 



Note: group numbers in this table refer to group numbers designated in Figure 1 



Conclusions 

On the posttest of far transfer, the metacognitive group scored significantly better (p 
<.001) than any of the other groups. In the within-group t test analysis, the metacognitive 
group was the only group which showed statistically significant increases from the pretest to 
the posttest on both near and far transier tests. This shows that the metacognitive 
question cues can positively affect the way in which students approach problems, and that 
this effect continues afterward for problems which are only distantly related to the training 
problem. 
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More evidence for this carryover of thought processes is revealed in the student 
interviews. When asked about the transfer tasks, students in the metacognitive group 
replied that they continued to ask themselves questions similar to the ones in the computer 
program, while the other cued groups did not. One control student, when asked about the 
post test answered " For the most part I knew I was right, so I did not question myself." 
Another student in the affective group, when asked if she asked herself anything similar to 
what she was asked on the computer program, replied "No. Not really. I did not really ask 
myself anything like that." During the interviews, the metacognitive students also spoke of 
the fact that if the cues had not been there, they would not have stopped to plan as well as 
they did, nor would they have stopped to contemplate why they got correct or incorrect 
answers. For example, one student when asked if she would do the same thing if she had 
not been cued replied "No. I wouldn't. I may just be satisfied with the fact that I got it." 
The same student, when asked if the questions affected her confidence, interest or 
motivations, said 'Tes. They made me feel good because I got them all right and I did the 
experiments right and understand the whole process better..." She stated that if she had 
not been asked the questions by the computer, she would not have felt the same way, 
"because those questions helped [her] to understand." 

There was variation within the groups, however, and some students replied that even 
though they were forced to give some reply to the cues, they did not give much thought to the 
cues. One student commented, "basically I just used pre-knowledge from the lab we 
originally did, the hands-on experiment. I used prior knowledge to decide what I was going 
to do next as I had originally done." One student's answers indicated that although she 
said she ignored the cues, she may have been using them. When asked if she had ignored 
the questions, this student replied, "Exactly. Those questions did not help me. But they did 
provoke me to think what I would do next. But I could not find the direct relationship 
between what I have thought and my understanding of the whole process" ( emphasis is 
authors'). All of the preceding quotes were from the students in the metacognitive group. 
Although the responses varied among these students, the majority of them (11/14) 
interviewed indicated that these cues caused them to stop, reflect and do something 
different than they would have without them. 

Although the qualitative analysis of students 1 task designs seems to contradict this 
"good news" by showing that, on average, the metacognitive group did worse on their task 
designs, we are reluctant to give these data heavy weight for several reasons. First, the 
designs were self reported by the students after they completed the computer activity. 
Within the "recorded" subset of students, when we compared their self-reports of their task 
designs with what actually happened while they were on the computers, 50% of the 
studonts either under or over represented the training task designs which they had actually 
completed. Second, after watching the screen recordings, we found that many of those 
students who were recorded also designed tasks which were not actually run, or changed the 
design of the task before they ran it. The students 1 self-reported data were not complete. 
The data tended to show only a segment of the task designs and did not accurately show 
the changes that students made during the process of designing the task. In analyzing 
these data, it is often not possible to tell if the students 1 diagrams represent an initial plan 
or the end result of the entire process of computer task design. It can be concluded from 
this experiment that retrospective qualitative data are not as accurate as on-line process 
data, and are not appropriate for collecting process data, especially when the data can not 
show the progress or changes made during the process. 

Therefore, even if the metacognitive group task designs were less comprehensive than 
those of the cognitive and control groups, we feel that this is not an indication that all of the 
cues failed (since the metacognitive group went on to outperform the other groups in the 
transfer test), but may mean that some of the cues failed. One cue that may not have been 
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successful for the metacognitive students' task design was the first cue ('What's your plan 
for solving the problem?"). This cue asked the students to detail their plans for the 
experiment. 

All of the students came to the experiment with strong preconceptions carried over from 
the experiences in the hands-on lab. Unless an event occurred during the process of the 
computer simulation to jar them out of this preconception, they continued to solve this 
problem as they did in the lab. For some students, reaching wrong answers or unexpected 
cues made them confront their preconceptions. Since almost 60% of the metacognitive group 
ended up in one of the two groups which only tested two variables independently (see Table 
5: 5&6), it may be that the cue which asked them "What's your plan for solving the problem?" 
was not different enough from what students were thinking at that moment to elicit a strong 
response and thus break them out of their preconceptions. 

This hypothesis is further supported by the interview data. When asked how they 
reacted to the first cue, most of the metacognitive students indicated that the main effect this 
cue had was to cause them to stop and think, or to slow down. For example, one student 
responded, "it made me think about it for a second before I took any actions. During that 
second it reminded me of what I did in class." However, while doing the hands-on lab 
experiment, the students were never required to test variables against each other. The 
problem-solving process they went through in the hands-on lab could not be applied to the 
problems posed in the computer program. Thus, the student's response was not appropriate. 
When asked if they would have done anything differently if they had not been given this cue, 
most of the metacognitive students answered that the only difference was that they would 
not have written down their plan, or that their plan might not have been so detailed. 
Another student responded "I put 'plan to test variable individually'. I did not make a very 
detailed plan." In all of these cases, we see that the first cue did not succeed in breaking 
these students' preconceptions, but allowed them to continue their task designs with their 
preconceptions intact. This might explain why so many of the metacognitive group had task 
designs which only tested the two variables (light and moisture) independently, and why they 
did not go on to test the two variables against each other. 

Contrasted to the metacognitive students' responses to the first cues, the cognitive 
students' interviews showed that in order to answer their first cue ("What variable(s) are 
you testing for?") they had to do a lot of metacognitive thought. For example, one student 
said, "They really made me think. They really did. They helped me along to see what I 
need to do in an experiment. They kind of making me think about exactly what variables I 
need to test and how many I am supposed to have in the experiment." Another student 
said, " They really made me think how to set up certain things. And it just made me think 
more consciously about what I was doing." This tells us that although the responses to this 
cue were cognitive 

(light, moisture, etc.), students had to go through a metacognitive process in order to 
respond to this cue appropriately. In fact, within the cognitive group, 9 of the 12 interviews 
indicated that the cognitive students went through such metacognitive-oriented processes 
when responding to the first cue. 

We can conclude that a cue which induces a metacognitive process can have a significant 
positive impact on students* problem solving processes. In the case of the first cue, the 
metacognitive response was actually found in the cognitive group. Consequently, their task 
designs were also much better than all the other groups. However, this single cue was not 
strong enough to guarantee that the rest of their thought processes continued to be 
metacognitively oriented, and the cognitive students failed to show far transfer. 

Even though the first cue in the metacognitive group failed to induce metacognitive 
processes, the metacognitive students subsequently received many more cues which actually 
induced metacognitive processes (refer to Table 4 for the category percentages for each 
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group). The high percentage of process responses generated by the metacognitive subjects 
suggest that the majority of the metacognitive cues successfully engaged students in 
metacognitive processing. This allowed the students to continue a process of "self- 
questioning" during the posttest, which may have helped them score significantly higher on 
the posttest of far transfer. Therefore, what students actually transferred were the 
metacognitive processes rather then the content-specific information. These processes made 
students focus on and become consciously aware of the solution components. By focusing on 
a process level, subjects were able to attend to information regarding the way in which they 
solved a problem. This finding is consistent with what Berardi-Colleta (1990) found in her 
study. She concluded that the process-focused groups are not only transferring knowledge to 
the new task, but that they are also transferring an approach to the new problem. 

Since all groups claimed to have planned, regardless of the type of cues, we propose that 
metacognitive planning needs to be carefully interpreted by the researcher. Mere planning 
was not sufficient to insure correct and complete task design. In order for task design to be 
correct and complete, these "plans" had to be coupled with active metacognitive processes 
such as monitoring, reflecting and evaluating. 

The issue of how these cues affect the students' task design process needs to be further 
explored. Given our experiences here, we would recommend that the process data be 
collected concurrently and be as detailed as possible. We would also caution that 
researchers and teachers need to be careful about their choices of cues, ensuring that the 
cues are attention focusing,and strong enough to break previously held preconceptions, as 
well as inducing metacognitive processes. Finally, the emphasis in problem solving research 
has placed on needs to shift to focus on human problem solvers as "memory banks" to 
humans as dynamic,active processors of their own learning experiences that are continually 
acquiring information in the learning process. 
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APPENDIX l:Cues Embedded Within the Computer Program 



Affective Group 


Cognitive Group 


Metacognitive Group 


Location One: after reviewing Paula's experiment and problem statements, and 

before any selections have been made 


Rationale: check feelings 
and mood before task 


Rationale: review the goals 
and rules of solving the 
problem 


Rationale: make plans for 
solving the problem 


(1) How are you feeling right 
now in dealing with this 
problem? 


(1) What variables are you 
testing? 


(1) What is your plan for 
solving the problem? 


Location Two: after selecting all of the materials and supplies, and before 

conducting the experiments 


Rationale: check on-task 
feelings 


Rationale: check actions 
against goals and rules of 
solving the problem 


Rationale:reflect on the 
actions have been taken and 
plan for the next step 


(2) How are you feeling 
right now? 


(2) What materials and 
supplies have you selected? 

(3) What are you going to 
test? 


(2) Why did you select 
those materials and 
supplies? 

(3) How are you deciding 
what to do next? 


Location Three: after finishing all of the experiments and drawing 

conclusions, and before the feedback was given 


Rationalexheck post-task 
feelings 


Rationale:encourage self- 
discovery of the current sate 
of the problem solving 


Rationale: monitor and 
evaluate the effectiveness of 
the process of the problem 
solving 


(3) How you feeling right 
now? 


(4) What conclusions can you 
draw about the variable(s)? 


(4) How did you decide that 
you have had enough data to 
make conclusions? 


Location Four: 


after providing the feedback 




ictationale:check the 
processes of the feeling 
change 


Rationale:check the final 
performance by knowing the 
number of the experiments 
that are useful or not useful 


Rationale:do not be 
satisfied with right or wrong 
answers, find out why they 
are right or wrong, build up 
a repertoire of strategies for 
further modifications of the 
processes 


(4) How are you feeling right 
now comparing to when you 
just got started? 


(5) How many of your 
experimental designs gave 
useful data? or 

How many of your 
experimental designs did not 
give useful data? 


(5) What part of your 
experiments led you to the 
right or wrong conclusions? 
*How are you going to fix 
it?(ask only if the conclusions 
are wrong.) 
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Appendix 2: Teacher-Made Near and Far Test of Problem Solving 



NAME. 



DIVISION 



BIOLOGY 205 LAB QUIZ FALL 1992 

INDIVIDUAL QUIZ - ISOPOD LAB 

1. What environmental conditions do isopods "prefer?" (4 pts.) 

2. Explain the basis for your conclusions about isopods' "preferred" environmental 

conditions. (6 pts.) 

3. Suppose you had the following information for the isopod lab: 

Ten isopods were placed in a trough in section 3, and after ten minutes, the following 
results were noted as diagrammed below: 
WET DRY 



* * * 



* 



1 2 3 4 5 6 

Based on the information given above, answer the following questions: 

a. What is the variable in this experiment? (1 pt.) 

b. How would you describe the response of these organisms to this variable (based on 
the diagram above) ? (2 pts.) 

c. What is random behavior? (1 pt.) 

d. Do you see random behavior in the diagram above? Explain vour answer . (3 pts.) 

e. What could you do to validate your results? (3 pts.) 



4. QUESTION (6 POINTS TOTAL) (FAR TRANSFER PROBLEM) 

The elementary school in which you teach seems to have an unusually high number of hyperactive 
students. In fact, of the 100 students in second grade, 50 of them are hyperactive. Different 
groups of adults have proposed various ideas to explain this. The parents say that the peer 
influence causes them to be hyperactive. The principal thinks that the family environment is 
the cause of their hyperactivity. The teachers, on the other hand, say that diet is the source of the 
condition. You have been asked to design an experiment that would determine which one or more 
than one of these explanations is correct. Explain how you would set up this experiment to 
determine the cause or causes of the students 1 hyperactivity. (Use the back of this sheet if you 
need more space.) 
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